In [1]:
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
from nilearn import plotting as ni_plt
import altair as alt
# Use this for large dataframes. But be smart!!
alt.data_transformers.disable_max_rows()
from altair import datum
from IPython.display import Image

import pandas as pd
import numpy as np

Exploring Brain Activity During Movement

For this exploration, I chose to look at brain data from my lab during arm movement. This data was collected from 12 ECoG patients undergoing epilepy monitoring at Harborview Medical Hospital in Seattle. Both video and brain data were recorded during their 7-9 day hospital stay. The video data was then used to generate data on movement events. Here, I explore what the brain signal looks like across patients, and on average, during different types of arm movements. The main questions I hope to answer are:

  1. What does the brain activity look like on average during arm movements?
  2. How does brain activity vary per person during arm movements?
  3. How does brain activity vary across different regions of the brain?
  4. How does brain activity change during specific types of arm movements, such as upward or downward movements?

More details about this dataset can be found in this paper: https://arxiv.org/abs/2001.08349

First, let us set some parameters and load in the data. These parameters will make more sense as we explore the data.

In [2]:
ROIs_of_interest = [47]
In [3]:
# movement_power_df = pd.read_csv('move_rest_avg_and_per_all_ROIs_power_spec.csv')
roi_pos_good_df = pd.read_csv('ROI_coords.csv')
hist_bin_df = pd.read_csv("reach_angle_histogram.csv")
reach_pca_time_df = pd.read_csv('reach_pca_time_direction_4components.csv')

1. What does brain activity look like on average during arm movements?

One way we can answer this question, is by looking at a spectrogram (visualization of the frequency spectrum through time) of brain activity during movement. This will show how different frequnecies may become more or less active when movement occurs

In [4]:
Image(filename='annotated_figures/avg_movement_annotated.png')
Out[4]:

Here we see a spectrogram that shows how the power (or activity) of different frequency bands during movement averaged across all participants. We can see an increase in power for high frequencies, and a decrease in lower frequencies, once movement is initated, as expected from the neuroscience literature.

2. How does brain activity vary per person during movement?

So far, we have seen the average brain activity during movement, but can we explore how different people's brains change during movement? To do this, we can create spectrograms for each participant.

In [5]:
Image(filename='annotated_figures/per_sbj_movement_annotated.png')
Out[5]:

We can see that participant electrode locations (places where brain activity was recorded) vary across participants, as well as the changes in the frequency bands. Surprisingly, participants with more motor coverage do not always show clearer signals in the frequency domain during movement, but Participant 5 shows a strong signal and has a good amount of motor coverage.

3. How does brain activity during movement vary across brain areas?

So far we have seen that brain activity tends to see increased power in high frequency bands, and decreased power in low frequencies during movement. But so far we have actually only looked at activity in one region of the brain! So where have we been exploring brain activity so far?

In [6]:
node_size = 50
sides_2_display = 'l'
num_electrodes = len(ROIs_of_interest)

colors = ['grey' for roi in roi_pos_good_df.iterrows()]
for i, c in enumerate(colors):
    if i in ROIs_of_interest:
        colors[i] = 'red'

fig,ax=plt.subplots(1,1, figsize=(7.5,5))
ax.set_title('Regions of Interest (ROIs) in Sensori-motor areas \n ROI 47 highlighted')
ni_plt.plot_connectome(np.eye(roi_pos_good_df.shape[0]), roi_pos_good_df, output_file=None,
                               node_kwargs={'alpha': 1, 'edgecolors': 'silver','linewidths':.5,'marker': 'o'},
                               node_size=node_size, node_color=colors,display_mode=sides_2_display, axes = ax)
ni_plt.show()

We can see that the part of the brain we were looking at activity from is located in the primary motor cortex. It is not surprising then that on average activity in this area changes with movement.

What other brain regions can we look at?

To explore how brain activity looks in different regions, lets pick a few different ROIs that cover different parts of the sensorimotor region to explore

In [7]:
ROIs_of_interest = [11, 47, 60, 123]
ROI_colors = {11:"aqua", 47:"red", 60: "blue", 123: "orange"}

colors = ['grey' for roi in roi_pos_good_df.iterrows()]
for i, c in enumerate(colors):
    if i in ROIs_of_interest:
        colors[i] = ROI_colors[i]

ROI_patches = []
for roi in ROIs_of_interest:       
    ROI_patches.append(mpatches.Patch(color=ROI_colors[roi], label='ROI '+str(roi)) )

fig,ax=plt.subplots(1,1, figsize=(7.5,5))
ax.set_title('ROIs to Explore in Sensorimotor areas')
ax.legend(handles=ROI_patches)
ni_plt.plot_connectome(np.eye(roi_pos_good_df.shape[0]), roi_pos_good_df, output_file=None,
                               node_kwargs={'alpha': 1, 'edgecolors': 'silver','linewidths':.5,'marker': 'o'},
                               node_size=node_size, node_color=colors,display_mode=sides_2_display, axes=ax)

fig.show()

Now that we've picked a few regions to explore, we can create new average spectrograms for each ROI and see how they compare

In [8]:
Image(filename='annotated_figures/per_ROI_movement_annotated.png')
Out[8]:

From this figure we can see that ROI 47 has the expected changes in high and low frequencies at movement initation, whereas the other ROIs are less clear. ROI 123 does seem to have an increase in power in high frequencies at least. Thus, it made sense that our data initally looked at ROI 47.

How does individual variance look in other sensorimotor areas?

To drill down a little deeper, we can also look at individual brain activity in a new area. Since ROI 123 showed the expected increase in power for high frequencies, we can look here.

In [9]:
Image(filename='annotated_figures/ROI123_per_sbj_power_annotated.png')
Out[9]:

Drilling down, we see that there is still a lot of variance in individual participant brain signals during movement, with Participant 0, 3, and 7 really driving the power increase in the higher frequencies that we saw. We also see that Participant 5 no longer has a very strong signal when we change the region we are looking at activity from.

4. How does brain activity change during specific types of arm movements, such as upward or downward movements?

So far, we have just looked at arm movements in general, but is this averaging hiding interesting activity that is happening for specific types of movements? To explore this further, we will dive into one aspect of the movements, reach angle.

First, lets explore what reach angles are most common in the dataset.

In [10]:
N = len(hist_bin_df['Bins'])
bottom = 0
max_height = 4

width = (2*np.pi) / N

ax = plt.subplot(111, polar=True)
bars = ax.bar(hist_bin_df['Bins'], hist_bin_df['Counts'], width=width, bottom=bottom)

# Use custom colors and opacity
for r, bar in zip(hist_bin_df['Bins'], bars):
    bar.set_facecolor(plt.cm.jet(r / 100.))
    bar.set_alpha(0.8)

plt.title('Histogram of Wrist Reach Angle for Movement Events \n Wrist Angle Based on Wrist Location at the End of the Reach Event \n')
plt.show()

We can see that a large majority of the wrist angles during these movement events are around 90 degrees, which makes sense considering that people often touch their face or use their hands for eating.

What does brain activity look like during different classes of movement?

Now that we have seen the distribution of the reach angles during movement, we can break them up into 4 classes (up, down, left and right movements) and explore brain activity during those states.

In [11]:
Image(filename='annotated_figures/movement_dir_power_annotated.png')
Out[11]:

We can see that upward movements have the strongest signal, with biggest decrease in low frequency power after movement initation. It turns out that this dataset was handpicked for clean, high signal movements, so it is not surprising that the best direction (up) also has the highest count of events.

How can we further visualize brain activity during movement?

So far, spectrograms of the brain activity during different movement classes do not show many differences. Is there a better way to visualize the data that will highlight differences in brain activity during different kinds of movements? To answer this question, lets try PCA on the brain data, reducing down the different movement direction events and seeing what is common across them.

In [12]:
dir_pca = alt.Chart(reach_pca_time_df).mark_circle().transform_filter(
    (datum.ROI == ROIs_of_interest[3])
).encode(
    alt.X(alt.repeat("column"), type='quantitative'),
    alt.Y(alt.repeat("row"), type='quantitative'),
    alt.Color('Direction:O', scale=alt.Scale(scheme='dark2'))
).properties(
    width=150,
    height=150
).repeat(
    row=['1st', '2nd', '3rd', '4th'],
    column=['1st', '2nd', '3rd', '4th']
).properties(title = "PCA Components of Brain Data from ROI 47 for Different Movement Directions")

dir_pca.configure_title(fontSize=20,anchor='middle')
Out[12]:

We can see in the 1st and 2nd components, the different directions create clear circular trajectories in the brain data. However, the other components provide less clear relationships.

How do these circular trajctories look through time and across ROIs?

Focusing on just the first two components, lets see how these circular trajectories move through time for each direction. Does there seem to be a relationship with the type of movement that occurs and where the trajectory starts and ends?

In [13]:
dir_roi_pca = alt.Chart(reach_pca_time_df).mark_circle().encode(
    alt.X('1st'),
    alt.Y('2nd'),
    alt.Color('Time', scale=alt.Scale(scheme='purples')),
    alt.Column('Direction', sort=['up', 'down', 'left', 'right']),
    alt.Row('ROI')
).properties(
    width=150,
    height=150
).properties(title = ["1st and 2nd PCA Components across ROIs and Through Time for Different Movement Directions",
                     "Time at 0 is when movement begins"])

dir_roi_pca.configure_title(fontSize=15,anchor='middle')
Out[13]:

We can see that unsurprisingly, the trajectories are clearer on the better ROIs (47 and 123). Interestingly, the different types of movements have quite different starting points (lighter purple dots), with opposite movements, such as up vs down, being near mirror images of each other.

Conclusions

After exploring this dataset, it seems that an increase in high frequency power, and/or decrease in low frequency power is quite common in movement. However, it is clear that averaging brain activity during movements hides which participants and what types of movements are really contributing to these trends in brain activity. Diving deeper, we see that some participants have really poor signal in some regions, but a clearer signal in others, such as participant 7. Upward movements also have a clear signal, which then likely contributed to the increased number of upward movements, as the dataset was hand-picked for clearer signals. After running PCA on the data, we seen an interesting circular trajectory appear, which the starting point and direction seem to have a clear tie to the movement type. This PCA embedding should be explored further to see how this relates to even more fine grained movement angles.